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ABSTRACT. Public agencies have started to shift away from the 
traditional lowest responsive and responsible bid to other approaches in 
purchasing certain items and services. These alternative approaches emphasize 
the quality of the products and the qualifications of the vendors. The purpose of 
this article is to explore the use qualifications-based selection (QBS) and other 
non-traditional source selection methods in public procurement processes. An 
online survey was sent out to 1665 members of the National Institute of 
Governmental Purchasing, and a mail survey sent a random sample to 300 
American Public Works Association members. The survey results show that 
while the traditional lowest responsive and responsible bidding is still the 
dominant selection method when all procurement is considered, QBS and other 
non-traditional methods have gained wide acceptance and use in public agencies, 
especially for the purchase of professional services and information technology. 


INTRODUCTION 


Public procurement is an old and a big business! (Thai & Grimm, 

2000; American Bar Association, 2000). But only until recent years 
have public officials realized their strategic role in contributing to the 
improvement of public services. For public procurement to meet such an 
important responsibility, governments have reengineered or reformed 
many aspects of the public procurement process. In this article the 
authors examine the changes in one of these aspects — how public 
agencies select vendors. 
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One emerging phenomenon in source selection is that public 
agencies have started to shift away from the traditional lowest responsive 
and responsible bid to other approaches in purchasing certain items and 
services, ranging from Request for Proposal (RFP) to qualifications- 
based selection (QBS). These alternative approaches emphasize the 
quality of the products and the qualifications of the vendors. The 
purpose of this article is to document and explore the degree to which 
QBS and other non-traditional selection methods are used in public 
procurement processes. As a pioneer study on the use of QBS, this 
research is important. Its results will allow both practitioners and 
academia to understand better selection practices of public procurement 
at the present and in the near future. To explore the use of qualifications- 
based selection in public procurement, this article has four parts. The 
first part reviews the literature regarding recent changes in public 
procurement and highlights the recent emphasis on qualification-based 
selection. The second part explains the research method. The third 
section discusses the findings in reference to the existing literature. The 
conclusion discusses the policy implications of this study and highlights 
future research. 


SOURCE SELECTION METHODS IN PUBLIC PROCUREMENT: A 
LITERATURE REVIEW 


With public demand for more effective and efficient services and with 
dramatic increasing use of information technology, public procurement 
has faced challenges and opportunities in recent years. Thai and Grimm 
(2000) noticed the environmental changes and the consequent issues 
public agencies have faced in recent years. Pettijohn and Qiao (2000) 
explored how public agencies have changed their ways to procure 
information technology (IT). McCue and Pitzer (2000, p. 400) stated 
that “in the face of growing uneasiness by elected officials, service 
delivery managers, and citizens about rule-driven process, inefficient 
systems, and poor management of resources, purchasing professionals 
are being challenged to develop new dynamic, adaptable structures.” 


Reforms in public procurement practices and process are inevitable, 
given the social and technological changes and the increasingly complex 
nature of public procurement. The traditional procurement system 
formed during the first part of the twentieth century is rules-driven and 
regulations-driven. These rules and regulations attempt to provide equity, 
integrity and efficiency in the public procurement system’ and worked 
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well in an environment in which government tasks were relatively simple 
and straightforward and when procurement dealt with simple 
commodities. However, as government jobs became more complex and 
its procurement, particularly the information technology and professional 
services acquisitions, became more intricate, many of the rules were 
outdated and severely limited the central procurement office's ability to 
respond to these changes (Kelman, 1990; National Association of State 
Procurement Officials [NASPO], 1999). 


Traditional Source Selection Method: Competitive Sealed Bidding 


While many changes have occurred in public procurement systems, 
the focus of this research is on how public procurement agencies select 
vendors and award contracts. “The evaluation of bids and offers is a 
highly significant part of the purchasing process. It brings together the 
two principal aims of public contracting: economy and _ faimess” 
(Council of State Governments, 1987, p. 72). Under the traditional 
procurement system is the long-standing policy recommendation that 
“competitive sealed bidding be the preferred method of contracting 
above a certain dollar amount” (NASPO, 1997, p. 1). All government 
procurement statutes require that under competitive sealed bidding, a bid 
is awarded to the lowest responsible and responsive bidder. 
“Responsiveness” means that the bid meets all of the material terms of 
the solicitation’s requirements. “Responsibility” means that the bidder 
“has the capability, financial capacity, and integrity to perform the 
contract” (NASPO, 1997, p. 66). 


The rise of competitive sealed bidding during the mid-20 century was 
necessary to prevent waste and corruption in public procurement. This 
practice opens the process to public scrutiny. All the sealed bids are 
opened publicly and the award goes to the lowest bidder, not to favored 
vendors. “Awards to other than the lowest bidder must be clearly 
explained in terms of the previous stated criteria” (Lynch, 1995, p. 331). 


Competitive sealed bidding worked well then, and it still works well 
now in determining awards when the product or services are essentially 
identical, irrespective of which bidder receives the award. However, 
there are situations where it is impossible to delineate adequately the 
differences among bidders when the award must be on a strict 
competitive bid basis. To address this inadequacy in the traditional bid 
system, some governments have enacted laws that grant procurement 
officials discretion to take action deemed to be most advantageous to the 
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jurisdiction, thus giving wide latitude of choice for the purchasing 
officials as long as they do not cross the line into arbitrary and capricious 
actions. This is particularly true with the purchases of IT and 
professional services where quality and performance of purchased items 
and services depend on the creativity or experience of vendors. The 
National Association of State Procurement Officials (1997) in its 
publication — State and Local Government Purchasing - and American 
Bar Association (2000) in its publication — Model Procurement Code for 
State and Local Governments — define this concept of “best interest” of 
the jurisdiction. Though the competitive sealed bidding can serve the 
best interest of the jurisdiction when procuring standard commodities, it 
is appropriate to say that the best interest concept encourages the use of 
alternative selection methods as reviewed below. For instance, Model 
Procurement Code for State and Local Governments indicates Request 
for Proposal and QBS will serve the best interest of the jurisdiction when 
competitive sealed bidding does not work. In addition to the “best 
interest” law, in the past two decades, many state and local governments 
specifically grant their procurement officials authority to use RFP, the 
two-step competitive bidding, the best value approach, and QBS that will 
be reviewed below. 


Alternatives to lowest responsible and responsive bidding selection 


In this section, we will review various alternative source selection 
methods, ranging from Request for Proposal to qualifications-based 
selection. While some of them such as Request for Proposal have been 
in use for a while, others like QBS are gaining more recognition and 
have been used only in certain areas. 


Request for Proposal 


The Request for Proposal is also known as the Competitive Sealed 
Proposal. While RFP often refers to an entire selection process, public 
works and infrastructure managers use the same term to refer to a distinct 
element (not the whole process) that may be used in competitive bidding, 
QBS, or other alternative forms of selection. 


As a selection process, the competitive sealed proposal is a complex 
procedure. It is used when competitive bidding is either impractical or is 
not advantageous to the public agency. One example is the procurement 
of an information technology commodity such as data processing 
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software. Procurement officers may not have the expertise to draft the 
specific or generic specifications, and it is unwise to use cost as the 
absolute criterion to evaluate the bids. Very often, state and local 
governments use the competitive sealed proposals method for the 
procurement of high technology commodities and professional services, 
especially data processing and telecommunication services (NASPO, 
1997). 


Like competitive sealed bidding, competitive sealed proposal also 
follows a competitive process but only in certain aspects. For instance, 
public procurement agencies have to announce competitive sealed 
proposals in public places. Under the competitive sealed proposal 
method, consideration is given to both price and other important factors 
such as experience, past performance, approach to the problem posed by 
the jurisdiction and staffing. However, price is not the most dominant 
factor, and negotiations are often encouraged between public 
procurement officers and venders (NASPO, 1997, p. 58). While 
competitive sealed proposals are opened publicly, usually only the names 
of the offerors are revealed. Whether other information is open to the 
public varies from jurisdiction to jurisdiction. Their prices and other 
information are not open to the public until public agencies make a 
notice of intent to award. 


Until 1979 when the American Bar Association (ABA) issued its 
Model Procurement Code for State and Local Government, competitive 
sealed bidding had been almost the only source selection method used by 
state and local governments (NASPO, 1997). Since then, many changes 
have occurred in public procurement due to the easy use of 
telecommunication technologies. To update its code that has been fully 
adopted by 15 state governments and thousands of local governments to 
the changing nature of public procurement and to offer the best practices 
to acquire the technology-oriented procurements, ABA published its 
revised Code in 2000 (Miller, 2002). The revised Code recognized that 
certain services may be better acquired in certain cases via the 
competitive sealed proposal process. 


According to the 1997-1998 Survey of State and Local Government 
Purchasing Practices (NASPO, 1999) forty-four states authorized its 
procurement officers to use competitive sealed proposals. Using 
competitive sealed proposal is not really a new practice. It has been in 
use for a while, though the survey did not document when these states 
authorized their agencies to use competitive sealed proposals. Twenty- 
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nine states reported that they utilize the competitive sealed proposals 
process in “some” solicitations issued; five states reported use of 
competitive sealed proposals in “most”; and twelve states described their 
use as “very few”. The same survey also found that nine states assign 
minimum weighting for prices. Twenty-three states reported that their 
use of competitive sealed proposal has some restrictions. Examples of 
the restrictions are “used for information technology and _ service 
contracting, and may be used for commodities under pilot projects” as in 
California and “must be approved by Secretary of Administration” as in 
Arizona (NASPO, 1999, p. 27). 


In some jurisdictions, procurement officials use a Request for 
Qualification solicitation to purchase professional services that are 
difficult to price. The process of Request for Qualification solicitation is 
similar to that of Competitive Sealed Proposal. The only difference is 
that price is not an evaluation criterion. Rather, the price or cost is 
negotiated between the procurement officer and the vender selected for 
contract award. 


Two-Step Competitive Sealed Bidding 


Two step competitive sealed bidding (also referred to as multi-step 
competitive sealed bidding) is a source selection method that allows 
public purchasers discretion to negotiate with possible vendors about the 
technical aspects of an acquisition but still requires the use of lowest 
responsive and responsible bidding to select the final vender. There are 
various versions of two-step competitive sealed bidding. One of the 
most advantageous versions works as follows: For the first round, 
potential bidders submit only technical proposals responding to the 
performance specifications issued by public agencies in an Invitation for 
Bid (IFB). Then, the evaluation team opens and evaluates the proposal 
and holds candid and detailed discussions with all the offerors, or only 
with those whose technical proposals have merit as to how well the 
proposal responds to the requests and how to improve the proposal. In 
the second round that follows the discussions, requests for bids are sent 
to those offerors with whom discussions have been held. The final 
award is determined mainly by the price. 


This method can be used to obtain the benefits of both competitive 
sealed bids and competitive sealed proposal procedures where the 
evaluation team can exercise discretion to negotiate and to discuss with 
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potential vendors. It “is particularly valuable in complex procurements 
when establishment of a full-blown statement of work or set of 
performance specifications is extremely difficult or not useful” (NASPO, 
1997, p. 60). For example, the state of Tennessee (Tennessee State 
Government, n.d.) requires its agencies to use two-step competitive 
bidding when acquiring institutional computer systems involving the 
purchase of hardware and the development of application software. 
Another advantage of this method is its reduction of the workload of both 
the bidders and procurement officers since only those that passed the first 
round screening will submit detailed bids in the second round. 


According to NASPO (1999), forty states have authorized its agencies 
to use this method. While twenty-six of them reported that their agencies 
use it “occasionally,” only nine indicated that they used it “regularly,” 
and five said they “never” used it (p. 62). National Performance Review 
(NPR) (1993) recommended a two-phase competitive source selection 
process for a design-build approach which “would focus on 
qualifications and concepts in the first phase and limit competition in the 
second phase to a specified number of contractors” (p, 57). In 1993, the 
U.S. General Service Administration started a two-step pilot program to 
select contractors for several federal design-build projects (“General 
Service Administration to Test,” 1993, p. 10). Such a two-step design- 
build selection method that emphasizes top-quality design for federal 
building projects has been institutionalized in U.S. General Service 
Administration’s design excellent program (General Service 
Administration, n.d.). 


Best Value Approach 


Another shift away from the traditional lowest responsive and 
responsible bidding is “the best value” approach that is mainly used in 
the procurement of information technology. It is also used for other 
acquisitions such as design-build projects. 


Kelman (1990) and NASPO (n.d.) attack use of the lowest responsive 
and responsible bid in procuring IT. Lowest responsible bidding for 
purchasing software or other information technology usually results in 
“product that is low in quality, high in risk and fails to meet the needs of 
the agency” (NASPO, n.d, p. 11). As a response, some governments 
allow their agencies to procure information technology based on best 
value rather than least cost. For IT procurement, non-cost factors such as 
performance of the vendor (e.g., reliability and the quality of services 
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and products) are more crucial than cost (Mechling, 1995; NASPO, n.d.). 
Several states, including Texas, Massachusetts, New York, Missouri, and 
New Mexico have adopted the best value approach (NASPO, n.d.). 
Gardner (1999), a Canadian writer, also reported that in recent years, 
government information systems management in Canada is increasingly 
awarding the business to those suppliers who bid solutions that 
dramatically increase employee productivity and customer satisfaction. 


The U.S. federal government also recognizes the need to use the 
“greatest value” or the “best value” approach in awarding contracts to IT 
vendors. “The award of a contract to a supplier based on lowest price 
can be a false economy if there is subsequent default, late delivery, or 
other unsatisfactory performance resulting in additional contractual or 
administrative costs” (National Performance Review, 1993, p. 51). In IT 
procurement, National Performance Review (1993) recommended that 
the negotiated method of procurement be used where “the government 
has considerable latitude in structuring the procurement and can consider 
both price and other factors (e.g., technical capabilities, qualifications of 
key personnel, past performance records, quality of proposed solutions) 
in selecting the contractor” (p. 52). The best value approach is also used 
in acquiring professional services to a large extent in the federal 
government (Office of Federal Procurement Policy, 1985). 


Best value is both a procurement philosophy and a source selection 
method. As a procurement philosophy, it emphasizes the overall return 
or value of the procurement rather than the lowest cost, as explained 
above. The RFP and the two-step competitive bidding that have 
reviewed above and the QBS to be reviewed below all fall under this 
philosophy. As a selection method, it is “a process used in competitive, 
negotiated contracting to select the most advantageous offer by 
evaluating and comparing factors in addition to cost or price” (Turbo 
Streamliner, 2000). Best value procurement has several underling tenets 
as outlined by the Professional Service Council (n.d.). 


- Statements of work that clearly reflect the specific (versus generic) 
requirements and the special quality demands of the customer and 
are expressed in terms of “what” is required rather than “how the 
work is performed.” 


- Formal and measurable (in terms of quality, timeliness, quantity, etc.) 
performance standards. 
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- Evaluation and selection procedures that utilize quality-related 
factors such as technical capacity, management capacity, cost 
realism, and past performance. 


- Incentive provisions to ensure reward for good performance based on 
predetermined performance standards. 


The best-value process has certain similarities to the QBS process to 
be reviewed below. Both processes emphasize the quality of the 
products or services, and both involve negotiation and allow procurement 
officials to use many non-cost factors as criteria in the selection process. 
However, they are different terms. The best value approach is a selection 
process that assesses the return “which can be achieved based on the total 
life cycle cost of the item. [It] can include an assessment of the 
functionality of the item; can use cost/benefit analysis to define the best 
combinations of quality, service and cost considerations over the useful 
life of the acquired item” (NIGP, 1996, p. 7). While the top priority is 
qualification in QBS, the best-value approach takes into consideration 
the combination of quality of the commodity (or qualification of vendors) 
and the costs. The point in best value procurement is that every penny 
should get its worth. There is another difference between the two 
approaches. Best value is a general process that usually involves the 
procurement of a commodity such as IT and professional services as well. 
QBS has a narrow connotation that often involves selection of vendors, 
particularly vendors that provide such professional services as 
engineering and architectural design. 


Qualifications-Based Selection 


Qualifications-based selection is a negotiated procurement process 
whereby service providers are selected on the basis of qualifications for a 
particular project, rather than price factors. The prospective vendor that 
has the best qualifications is offered the contract, and fee is considered 
and negotiated only after selection. While some website and literature 
describes the QBS process in more details than others and the specific 
sequences may be a little different, the major elements in QBS process 
remains the same as highlighted in Table 1. 


As indicated above, under QBS, professional service firms are 
weighted first on competence, creativity and performance, and second on 
negotiation of a fair and reasonable fee (Indiana QBS Coalition, n.d.). 
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TABLE 1 
QBS Process 


Step 1: Select the best | -Establish evaluate criteria 

qualified firm -Solicit statements of qualifications from 
interested firms 

-Develop a short list of 3 to 5 firms. Investigate 
references, jobs and office. Invite for interview 
-Interview and rank the firms 

Step 2: Jointly define -Invite the highest-ranked firm to assist in 
scope and contract terms defining the scope of the work. 

-Establish contract terms 

-If the agency and firm do not agree upon the 
terms, go to the next firm on the short list 

Step 3: If yes, retain the | -Ask for fee proposal 

firm on the basis of an | -Agreement? If not, go to the next firm on the 
acceptable proposal short list; if yes, retain the firm and enter into a 
written contract 


Source: QBSColorado (n. d.). 


Like the traditional competitive sealed bidding, QBS also involves 
competition. However, the competition here is on a different basis — that 
of qualification. To assess the qualifications of a firm, interview and 
discussions, not bidding, are needed. 


While public agencies have experienced an increasing use of QBS in 
recent years, there is no academic research about its use. In addition to a 
number of websites sponsored by professional associations promoting 
the use of QBS, the authors are fortunate to have obtained a series of 
leaflets, handouts and publications on the use of QBS from American 
Public Works Association (APWA). According to the limited available 
literature, qualifications-based selection is mainly used in selection of 
professional services of engineering, architecture, and land survey. 


Using OBS for Professional Services 


As early as 1987, NASPO recognized the controversy over the use of 
sealed competitive bidding to select professionlal services.’ “Some 
professional groups argue that their fees should be non-competitive and 
that price should not be a factor in evaluating proposals for their 
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services” (Council of State Government, 1987, p. 68). However, concern 
exists over whether the noncompetitive and non-fee-based approach 
would result in monopoly and corruption. NASPO (1997) also 
emphasized the complex and demanding nature of procurement of 
professional services. “While the purchase of services requires 
innovative methods of acquiring quality services at a reasonable price, 
traditional procurement strategies are amenable to the process” (p. 74). 
As indicated above, in most cases, professional services are procured 
through competitive sealed proposals, although the option of multi-step 
competitive sealed bidding is also considered (NASPO, 1997). 


It is interesting to note that the public procurement community and 
private industries hold different views as to whether QBS or 
conventional selection methods should be used. This difference was well 
documented in 1985 report by the Office of Federal Procurement Policy 
(1985, p. vii): 


Contracting for professional services is viewed differently by the 
government than by certain private organizations and interests. 
The government contracting community is, for the most part, 
satisfied with the present contracting process.’ Contracting 
officials stated that current procedures provide for the proper 
consideration of both cost and technical factors and that contract 
awards can now be made on the basis of technical and quality 
factors when appropriate to do so. Private trade associations and 
organizations, representing surveyors, mappers, real estate 
appraisers, and soils consultants, are not satisfied with present 
practices. The surveyors, mappers, and soils consultants want the 
services offered by their respective professions to be acquired 
pursuant to the contracting process specified in Public Law 92-52, 
the “Brooks Architect-Engineer Act.” 


In the past few decades, there has been an increase of interest and use 
of QBS in procuring professional services in the areas of engineering, 
architecture, and land surveys.” A number of professional associations 
such as the American Public Works Association (APWA), the American 
Consulting Engineers Council (ACEC), the American Institute of 
Architects (AIA), and the National Society of Professional Engineers 
(NSPE) are strong advocators and promoters of the use of QBS.° They 
have identified several rationales for the use of QBS in procuring 
engineering, architecture, and land survey. 
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First, QBS is the appropriate method in selection vendors for the 
creative professional services in that creative services cannot be 
described precisely. “Qualifications for architectural and engineering 
services, like those for medicine and law, do not lend themselves to 
being set forth in detail in advance as is done for construction services 
and office supplies” (Martin, 1997, p. 2). Therefore, “reliance must be 
placed on the experience, expertise, creativity and overall intellectual 
capacity of the people involved who will ultimately determine the 
success of the project design or technical study” (Martin, 1997, p. 1-2). 
Bidding or other cost-based selection does not produce the best-qualified 
firms for the job, because lowest cost and high quality never go hand in 
hand. No two firms can or will have precisely the same qualifications for 
any particular projects design or technical study. Only detailed 
interviews allow public agencies to evaluate a technical consultant’s 
qualification related to the work at hand, and negotiating a detailed scope 
of work with the highest ranked firm under QBS serves as a basis for 
realistic fees and promotes full cooperation of the consultant in fulfilling 
the contract (Martin, 1997). 


Second, while design fees are always a very small part of overall 
project costs, regardless of the method of consultant selection, quality of 
design has a profound influence on the life-cycle cost of the project. As 
the first step and also the most important element in any construction 
project, planning and designing determines the size of the layout of the 
facility; type of construction materials; capacity of mechanical and 
electrical systems; energy efficiency; and other factors” (QBS Utah, n. 
d.). When a defect occurs in the design, even the best contractor using 
the finest construction materials cannot overcome the effects. Indeed, 
compared with construction and the life-cycle costs, the cost of planning 
and designing is very small. The federal government estimated that 
original design service costs is only about 1% or less of the project’s life- 
cycle costs. Therefore, choosing the best designers will have long-term 
economic benefits. According to Martin (1997), QBS can achieve the 
goals of economy, safety, efficiency, sound construction, serviceability, 
maintenance, and operations. 


Third, QBS facilitated by interview and negotiation fosters a 
partnership relation between the clients and the design firms. This type 
of relation will allow design firms to play an effective and constructive 
role in representing the owner’s interests in day-to-day dealings with 
contractors, suppliers, equipment manufacturers, and others providing 
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goods and services during construction (QBS Utah, n. d.). QBS also 
fosters greater creativity and flexibility, and minimizes the potential for 
disputes and litigation. Overall, “the public interest is best served when 
governmental agencies select architects, engineers, and _ related 
professional technical consultants for projects and studies through 
Qualifications Based Selection (QBS) procedures” (Martin, 1997, p.1). 


Legal Bases for OBS Use 


There are some legal bases for the use of QBS in public procurement. 
The federal government passed the Brooks Act in 1972 (Office of 
Federal Procurement Policy, 1985). Congress mandated a policy of 
negotiating contracts for architectural and engineering services on 
government projects on the basis of "demonstrated competence and 
qualification ... at fair and reasonable prices" (cited in Office of Federal 
Procurement Policy, 1985, p. 10). Since then, many states and numerous 
localities have followed the federal order. According to American 
Consulting Engineers Council’s (ACEC) 2001 survey, 42 states passed 
legislation to authorize the use of qualification-based selection in their 
states. The specific requirements differ from state to state. For instance, 
in Maryland, under $100,000 value of purchase price is 40% of selection 
criteria. Tennessee requires qualifications but also allows considering 
prices when its local units use QBS. Thirty-seven states, and local units 
in 21 states were allowed to use QBS (American Council of Engineering 
Companies, n.d.). 


Professional associations have made constant efforts to promote the 
use of QBS. For instance, NSPE and ACEC have co-sponsored national 
qualifications-based selection awards for both the government sector and 
the non-government sector to honor entities that use QBS to retain design 
professionals. Since 1989, ACEC, AIA, APWA, and the National 
Society of Professional Engineers/Professional Engineers in Private 
Practice (NSPE/PEPP) have funded and administered a QBS Facilitator 
Grant Program. Modeled after the remarkable success of the Wisconsin 
State QBS effort that started in the mid-1980s, the program is to assist 
state efforts to inform public agencies about the QBS process (Martin, 
1997). “State programs, which receive the grants, are expected to assign 
or engage ‘facilitators’ to encourage selection of A-Es on the basis of 
qualifications, rather than price” (Ross, 2000). These national 
associations and many regional associations have distributed various 
publications, workbook, handouts, and websites to assist public agencies 
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in understanding the importance of QBS and how to successfully 
implement a recruitment based on qualifications. 


In addition, the American Bar Association also recommended state 
and local governments “negotiate contracts for Architectural and 
Engineering Services on the basis on demonstrated competence and 
qualification for the type of services required, and at fair and reasonable 
prices” (American Bar Association, 2000, p. 48). 


Research Questions 


The above literature review shows that due to the increasing 
complexity of commodities and services, public agencies have started to 
use various source selection methods in addition to the traditional lowest 
responsive and responsible bidding method. It seems that most of the 
innovative approaches are used mainly in the acquisition of IT and 
professional services, including human services, engineering, and land 
surveys. However, we do not know to what extent QBS and other 
alternative approaches are used in these acquisitions, how they are used 
and the impact of the uses. Therefore, this study will focus on the 
following research questions regarding the use of QBS and the 
alternative approaches: 


1.To what extent are QBS and other non-traditional selection methods 
used in public procurement, in what areas and in what forms? What 
are the factors that explain the use in some jurisdictions but not in 
others? 


2.1f QBS is used, are public agencies happy with it? Does it work 
better than the traditional lowest responsive bidding in that specific 
acquisition area? If QBS is not used, why? 


3.What is the trend? Do those that are not using QBS plan to use it in 
the near future? 


METHODOLOGY 


To document the use of qualifications-based selection process, 
questionnaires were sent to about 1665 National Institute of Government 
Purchasing (NIGP) members throughout the United States and Canada 
through e-mails and to 300 American Public Works Association 
members by mail in December 2001. Responses from the online NIGP 
survey totaled 216. However, seven pairs and three clusters, each of 
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which has three cases, have identical responses. The authors suspect 
those answering the survey may have sent their responses twice or three 
times. If this is true, the number of online surveys is reduced by 13 to 
203. However, all of these responses were kept in the analysis for two 
reasons: First, the authors were not 100 percent sure that the duplication 
had, in fact, occurred. Second, online surveys that may have been 
submitted multiple times is small and will not distort the analysis results. 


Most of the online responses came in December 2001 and early 
January 2002. The last response was in the middle of February 2002. 
There are 87 responses from American Public Works Association 
members, a response rate of 29%. All of the responses came between 
December 2001 and February 2002. 


The two versions of surveys were slightly different. The survey 
instrument completed by NIGP members consists of an online survey, 
which was designed and sent to the NIGP office in summer 2001. When 
the survey instrument was sent to Dennis H. Ross, Director of 
Professional Development for the American Public Works Association, 
he made valuable comments. Accordingly, several survey questions’ 
were revised. In addition, the online survey instrument completed by 
NIGP members cannot collect data on some questions.’ The survey 
completed by APWA members, a mailed survey with a pre-addressed 
return envelope, was designed to allow more detailed exploration. The 
differences in the questions on the two surveys are explained in the 
discussion of each question. 


FINDINGS 


The findings cover the use of QBS in public procurement from 
various aspects, including what percentage of the respondents are 
allowed to use and do use QBS, how they use it, whether they are 
satisfied with it, why they do not use it if they have not used it, and 
projected use in the coming 3-4 years. 


Number of Public Agencies Using QBS 


One survey question asks respondents which of the following 
selection methods is allowable under their enabling legislation: The use 
of QBS; selection of a vendor based on “best interest of the 
jurisdiction(s); or the use of lowest responsive and responsible bidder 
only. As Table 2 shows, about one third (e.g., 29.5% of the APWA 
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respondents and 32.4% of the NIGP respondents) indicated the laws 
allow them to use QBS. The percentage of APWA respondents that are 
allowed to use QBS can be as high as 59.3%, because another 29.8 
percent indicated that they are allowed to use a combination of the three 
methods (e.g., 8 percent stated that they are allowed to use both QBS and 
best interest, 9% are allowed to use QBS, best interest, and the lowest 
responsible and responsive bidding, and 14% are allowed to use both 
QBS and the lowest responsive and responsible bidding). Whether the 
NIGP respondents are also allowed to use a combination of these 
methods is not known, because the coding of the online survey allows 
only one choice while mail survey respondents can mark multiple 
choices. 


While the existing literature reveals that by 2001 thirty-seven states 
had authorized the use of QBS for their state contracts and twenty-one 
states allowed their local units to use QBS, the finding of this study 
is that about one-third to half of the respondents confirmed the authority 


TABLE 2 
Selection Methods Allowed by Enabling Legislation 
APWA NIGP 
N=87 N=216 
Qualifications-based selection 29.5% 32.4% 
“Best interest of the jurisdiction’”* 18.2% 29.2% 
Lowest responsible and responsive bidding 15.9% 34.7% 
A combination of QBS & other method(s) 29.8% N/A 
A combination of “best interest” with others | 17.6% N/A 
Missing cases 2.3% 3.7% 
Total 96%** 100% 


Notes: 


* “The best interest of the jurisdiction” is a broad concept as explained in the 
literature review. As explained in the NIGP (1996), in the absence of 
specific authority, “best interest” law provides an official discretion in 
taking the action deemed to be most advantageous to the jurisdiction” (p. 7). 
It encourages a wide range of selection methods, particularly those non- 
traditional selection methods such as RPF, best value approach and QBS. 


** The 4% that is not shown in this calculation is the cases where respondents 
marked “best interest of the jurisdiction” together with the lowest 
responsive and responsible bidding. 
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to use QBS. While these numbers do not agree exactly, they may be 
explained by the sample bias in which there are few state respondents 
and many representing city and county governments. 


As for the alternative enabling legislation, 15.9% of the APWA 
respondents and 34.7% of NIGP respondents indicate they are required 
to use the lowest responsible and responsible bidding. Though 18.2% of 
the APWA respondents stated that they are authorized to select a vendor 
based on best interest of the jurisdiction, another 17.6% of them 
indicated they could use both “best interest” together with QBS or/and 
lowest responsive and responsible bidding. Twenty-nine percent of the 
NIGP group is authorized to use best interest of the jurisdictions. 


It is interesting to observe that the two groups have a like proportion 
of respondents governed by the “best interest” laws. Higher proportions 
of NIGP respondents are allowed only to use “lowest responsive and 
responsible” law, but higher proportions APWA respondents are 
authorized by the QBS laws. 


Then how widely is QBS actually used by the respondents? The 
findings are reported in Table 3. Fifty-six percent of the APWA group 
and 73% of the NIGP respondents are selecting vendors based on 
qualification; Though the lowest responsive and responsible bidding is 
still more widely used (91% of APWA members and 61% of NIGP 
members) than vendors’ qualification, we can say that QBS has gained a 
wide use. This conclusion is also supported by the fact that another 14% 
of AWPA and 32% of NIGP respondents are using a hybrid method in 
which qualification or quality is often considered. The finding that 91% 
of APWA respondents are using lowest responsible and responsive bid 
may reflect their construction contracts rather than professional service 
contracts. 


Many of the respondents stated that they are using more than one 
selection method. For instance, 64% of the APWA group and 56% of 
NIGP group stated that they select vendors on the basis of both vendors’ 
qualification and quality of the commodities. Around 9% of APWA and 
14% of NIGP respondents state they use a combination of qualification, 
quality, and hybrid. It is also interesting to find out that 37% of APWA 
and 38% of NIGP respondents are using a combination of price, 
qualification, and quality. This may prove that NIGP members are 
allowed to use multiple selection methods, and the online survey did not 
pick up this information as explained in the previous paragraph. 
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TABLE 3 
Vendor Selection Criteria 
Selection criteria APWA members |NIGP members 
N=87 N=216 

Lowest responsible & responsive bid 91% 61% 
Quality of commodity 44% 61% 
Vendor’s Qualification 56% 73% 
Hybrid 14% 32% 
Others 2.% 27% 
Combination of quality & qualification 64% 56% 
Combination of quality, qualification, 9% 14% 

& hybrid 
Combination of price, quality, and 37% 38% 

qualification 


Crosstabulation between Enabling Legislation and the Use of QBS 


The authors did a cross-tabulation with selection methods and the 
actual QBS users, and the findings are reported in Table 4. As expected, 
QBS users heavily concentrated in those jurisdictions that have QBS 
enabling legislations, which accounts for one third of the respondents. 
Twenty-one percent of respondents indicated that they are allowed to 
select vendors on the basis of best interest of the jurisdiction and they use 
QBS. It is interesting to observe that while 8.1 percent of the respondents 
are allowed to use QBS under the law, but do not use QBS methods, 16% 
of the respondents that use some form of QBS method state that their 
enabling legislation requires all purchases based on lowest responsive 
and responsible bidder only. How are these contradictions explained? It 
may be the way the survey is coded. Though the online survey allows 
the respondents to check only one of the three types of enabling 
legislation, respondents may be allowed to use multiple methods. This is 
obvious in the mail survey where many respondents checked multiple 
methods. Some online respondents who checked the “lowest responsive 
and responsible bid,” which may be the dominant selection method, may 
also be allowed to use QBS to a certain extent. Whether the law allows 
public agencies to use QBS methods is statistically significant in 
explaining whether QBS methods are adopted as seen by the significant 
level of .000. 
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TABLE 4 
Crosstabulation between Enabling Legislation and the Actual Use of 
QBS (N=303) 
Selection methods allowable under your enabling legislation 

QBS For “best interest |Lowest responsible | Total 

methods _|of the jurisdiction” | & responsive only 
Use QBS* = |33.3% 21.8% 15.8% 70.9% 
Not use QBS* |8.1% 5.6% 15.4% 29.1% 
Total 41.4% 27.4% 31.2% 100% 
Df=2 
Pearson Chi-Square: 25.899 
Significance level = .000 


Note: * Use or Not Use of QBS is based on the answers to the question — 
Vendors are selected by a) lowest responsive and responsible bids, b) 
quality of the commodity, c) qualification of vendors, d) hybrid method, 
and e) others. If the respondents report they select vendors by qualification 
of vendors, then they are coded as “Use of QBS”. 


The Distribution of QBS Users 


The authors did various crosstabulations to illustrate the distribution 
of the QBS users. Table 5 reports the distribution of the QBS users by 
regions. Among the 71% of the respondents that use QBS, 25% of them 
are in the South, 17% in the West, and about 12% in Midwest and 
Northeast respectively. Table 5 also reports QBS users v. non-users 
ratios by regions. As shown, the South has the highest users v. non-users 
ratios. There are four QBS users for any one non-user. The lowest user 
v. non-user ratio within the U.S. is found in the Midwest. The region has 
been found to be a statistically significant in explaining the use of QBS. 


Table 6 reports the distribution of QBS users by the size of 
community’s population. The QBS users concentrate in medium and 
large cities. For instance, over 43% of all the respondents that use QBS 
are in the communities with population larger than 100,000 (e.g., 20.5% 
for communities with population of 100,000 —300,000 and 22.4% for 
communities with population over 300,000). Table 6 also shows that 
those communities with population of 50,000-100,000 and 100,000- 
300,000 have the highest QBS users v non-users ratios. But the sizes of 
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the cities are not statistically significant in explaining whether QBS is 


used or not. 
TABLE 5 
Distribution of QBS Users by Regions 
Regions Total 
Midwest |Northeast |South | West |Outside | Missing 
USA 

QBS users |12.9% 12% 25% 17% |4.3 3% 71% 
Non-users {10.6% 5.0% 5.3% [5.3% |2.6% —_|.7% 29% 
Total 23% 17% 30% —|22% |7% 1% 100% 
Ratio* 1.2/1 2.4/1 4.7/1 3.1/1 | 1.7/1 1/2.3 2.4/1 
N 71 51 91 66 {21 3 303 
Df. =6 
Pearson Chi-Square: 18.16 
Significance level = .006 


Note: * QBS user v. nonuser ratios = the percentage of those that use QBS is 
divided by percentage of those that do not use QBS. For instance, QBS 
users V. non-users ratio for the Midwest = 12.9/10.6 = 1.2:1. 


TABLE 6 
Distribution of QBS Users by Size of Community’s Population 
(In 1,000) 
Population of communities Total 
<10 10-50 | 50-100 | 100-300 |>300 | Missing 
QBS Users | 6.3% |11.9% | 10.9% |20.5% 22.4% |.7% 71% 
Non-users [4.0% |6.9% [4.0% |8.3% 13.1% [3% 29% 
Total 10% 18.8% |14.9% |28.7% 35.5% | 1.0% 100% 
Ratio 16/1 [1.7/1 | 2.7/1 2.5/1 17/1 {2.3/1 2.4/1 
N 31 60 45 87 108 3 303 
Df. =5 
Person Chi-Square = 4.856 
Significance level = .434 
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Table 7 reports the distribution of QBS users by the size of 
professional employees in the department. It is interesting to note that 
39% of all respondents that use QBS have only a small number of 
professional employees (5 or fewer). This group is also the leading non- 
users. Only 26% of the respondents that use QBS have more than 6 
professional employees. Those that have 6-10 professional employees in 
the department have the highest users v non-users ratios. Number of 
professional employees is not a statistical factor in explaining the use of 
QBS. 


TABLE 7 
Distribution of QBS Users in Terms of Number of Professional 
Employees in the Department 


Number of professional employees in the Department 


None <5 6-10  |>10 Missing | Total 
QBS Users 5.6% 38.6% |12.9% | 12.9% | .7% 70.6% 
Non-users 3.0% 15.5% [3.6% | 5.6% | 1.7% 29.4% 
Total 8.6% 54.1% |16.5% [18.5% |2.3% 100% 
Ratio 1.9/1 2.5/1 [3.6/1 {2.3/1 0.4/1 2.4/1 
N 26 164 50 56 7 303 
Df. =4 


Pearson Chi-Square = 7.689 
Significance level = .104 


Table 8 reports the cross-tabulation between QBS users and number 
of professional employees in the entire agencies. The question about the 
number of professional employees in the entire agency is sent only to 
APWA members. The finding shown in Table 8 is consistent with that in 
Table 7. Most of QBS users (24%) have a relatively small number of 
professional employees in the entire agencies (e.g., with 5 or fewer 
employees). Again the number of the employees in the entire agency is 
not a statistical significant in explaining the use of QBS. 


Table 9 reports the correlation between types of jurisdiction and QBS 
users. Almost half of QBS users are in the cities, but cities are also the 
leading non-QBS user. The QBS user v. non-user ratio is 2/1. Counties 
and state agencies as a whole rank second as QBS users. 
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TABLE 8 
Distribution of QBS Users by Number of Professional Employees in 
the Entire Agency 


Number of professional employees in the entire agency 
None | 5or less | 6-10 10-15 | >15 Missing | Total 


QBS user | 9% 24% 8% 2% 11% | 9.1% 64% 


Non-users_ | 6% 17.0% 5.7% | 0% 4.5% | 3.4% 36% 


Total 15% | 41% 14% | 2% 16% 13% 100% 
Ratio 1.6:1 | 14:1 14:1 | N/A [| 2.5: 1 | 2.751 1.7: 1 
N 13 35 12 2 14 12 87 
Pearson Chi-Square = 2.511 

Df=5 


Significance level = .755 


TABLE 9 
Distribution of QBS Users by Types of Jurisdictions 


Types of Jurisdictions 
City |County |State |School |Special |Federal |Other | Total 
District | District 


QBS users | 33% | 15% 14% |4% 1% 71% 3.0% |71% 
Non-users |16% |5% 5% |1% 3% None |1% 29% 


Total 49% |19.8% |19% |5% 1.3% |.7% 4% 100% 
Ratio 2/1 {2.7/1 | 2.5/1 | 4.3/1 | 3/1 N/A 2.3/1 [2.4/1 
N 148 |60 57 16 4 2 13 303 
Pearson Chi Square = 3.504 

Df. =8 


Significance Level = .899 


How to Use QBS 


This last section documents QBS users and distribution. This section 
reports how QBS and other non-traditional source selection methods are 
used from several aspects. 


Different Non-Traditional Source Methods 


First, what non-traditional source selection methods (e.g., RFP, two- 
step competitive bidding, invitation to negotiation, letter of interest, best 
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value/best delivery approach, Brooks bill approach) do public agencies 
use? According to the findings reported in Table 10, the most frequently 
used method is RFP (93% among the APWA group and 87% among the 
NIGP). About a quarter of APWA members are using invitation to 
negotiation and two-step competitive bid, and about a quarter of NIGP 
members are using QBS, letter of interest, and best value approach. 
While higher proportions of APWA respondents are using letter of 
interest (31%) and invitation to negotiation (26%) than the NIGP 
members (13% and 24% and 13%), more NIGP respondents are using 
best value approach. Only 3% in both groups indicated they are using 
Brooks bill approach. 


TABLE 10 
Use of Different QBS Approaches 
APWA members NIGP members 
(N=87) (N=216) 
QBS N/A 26% 
RFP 93% 87% 
Two step competitive bid 23% N/A 
Invitation to negotiation 26% 13% 
Letter of interest 31% 24% 
Best value approach 19% 27% 
Brooks bill approach 3% 3% 
Others 2% 9% 


How often do the agencies combine non-traditional methods? As 
Table 11 shows, while 39% of all the respondents use only one of the 
approaches (e.g., RFP), the survey does find many public agencies using 
a combination of these non-traditional approaches. For instance, 27% 
reply that they use two of them, and 12.5% are using three of them, and 
8% are using four of these approaches. Only 1% combines all six non- 
traditional approaches. 


Extent of Using OBS 


Another question asks the extent agencies have used QBS in their 
procurement. This is another question potentially affected by the 
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TABLE 11 
Combinations of QBS Approaches Used by Public Agencies 
Various combinations of QBS methods | % of the overall respondents 
N= 303 

1 approach 38.9% 

2 approaches combined 27.7% 

3 approaches combined 12.5% 

4 approaches combined 8.3% 

5 approaches combined 2.6% 

6 approaches combined 1.0% 


different versions sent to the two different groups. The questionnaire 
sent to NIGP members is a general one asking “to what extent do you use 
QBS where 1 means ‘none’ and 5 means ‘for all purchases’”. The 
findings are reported in Table 12. Fourteen percent do not use QBS at 
all and 6% use QBS for all the purchases. Many indicate (40%) that they 
use QBS only to a limited extent while about 27% of the respondents 
reply they use QBS for half of their purchases. This blanket result does 
not show what purchase they use QBS for. The questionnaire sent to 
APWA members fills this gap by breaking down the purchases into 
different categories and asking respondents to mark the extent they use 
QBS in each type of purchase (e.g., commodities, professional services, 
vocational rehabilitation, contractual technology, construction contracts). 
The findings are reported in Table 13. 


TABLE 12 
To What Extent Do Public Agencies Use QBS: Responding from 
NIGP Members 
(1 means “none” and 5 means “for all purchases”’) 
N= 216 1 2 3 4 5 | Missing data 
% NIGP members 13.9 | 40.3 26.9 6.0 5.6 74 


As shown in Table 13, QBS is more often used in purchasing 
professional services followed by procurement of IT. Fourteen percent 
of respondents use QBS for half of their professional services and 27% 
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TABLE 13 
Extent of Use QBS: % among APWA Responses 
N=87 
Types of purchases 1 |2 3 4 5t Missing 
Commodities/supplies 38.6 {19.3 |10.2 [6.8 [4.5 [20.5 


Professional services (e.g., A/E, |6.8 |2.3 |13.6 |27 |40.9 |9.1 
accounting client service 
contract) 


Vocational rehabilitation, 21.6 |2.3 110.2 |6.8 [5.7 53.4 
guidance counseling, 
employment services 


Contractual services (e.g., lawn | 20.5 |18.2 |28.4 |2.3) |11.4 |19.3 
care, janitorial, computer 


maintenance) 
Information technology 14.8 {8.0 [19.3 [114 [11.4 [35.2 
Construction contracting 42.0 |17.0 |8.0 |9.1 [11.4 |12.5 


Notes: * “1” means “none” and “5” means “for all purchases.” 


uses QBS for more than half of their professional services; and 41% use 
QBS for all of their professional services. There is a little use of QBS for 
purchasing commodities and supplies and construction contracting. 
However, it is interesting to observe that QBS is not always used with 
the purchasing of professional services and QBS is used for commodities 
and supplies to a limited extent. The purchase of information technology 
has a relatively even spread as to what extent QBS is used. Forty-two 
percent of construction contracts do not use QBS at all. 


Required Information in QBS Selection Process 


As shown in Table 14, all of the information is required when public 
agencies select vendors through the QBS process. However, the most 
frequently required information is the proposer’s ability to perform a 
function, which is required by 87% of NIGP respondents and 97% of 
APWA respondents. While a higher proportion of the APWA group 
require experience of staff and experience of the company, more NIGP 
members require financial stability of the vendors and understanding of 
the project among the areas, vendor’s demonstration, sites visits, and 
vendor’s approach to the problem or task. Both groups also require other 
information, including availability of experienced staff and fees, cost 
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after being accepted, experience in energy reduction initiatives, fee 
compensation proposal, proof of insurance, prevailing wages, 
certification, past experience with the companies, cost for those who are 
on the short list, certifications of staff, criminal record checks for 
sensitive areas access and whether they are bondable, financial rating, 
value added incentives, general marketing plan, interviewing with 
company, meeting bonding requirement, product or service availability, 
quality assurance program, quality of the response to RFP, safety policies 
and procedures, occupational health and safety clearances of staff, 
sample, DOT pre-qualification, total long term cost to acquire goods or 
services. NIGP members also indicated that their requirements differ 
from the type of purchases. 


TABLE 14 
Information Required in the QBS Selection Process 
Information required % APWA | % NIGP 
responses | responses 
Proposer’s ability to perform a function 96.6 87.5 
Proposer’s staff names 59.1 67.1 
Experience of its staff 90.9 77.3 
Experience of the company 93.2 84.3 
Financial stability of the vendor & understanding 61.4 81.0 
of the project among the areas 
Vendor’s demonstration 36.4 61.6 
References 81.8 84.3 
Site visits 39.8 64.8 
Vendor’s approach to the problem or task 63.6 71.3 
Others 7.9 13.4 


Table 15 shows that when using the QBS method to select vendors, 
over 70% (e.g. the sum of 13.5%, 15.8%, 13.9% and 31.9%) of the 
respondents require 6-9 types of the information, and 32% indicated they 
require all of the nine types of information listed. A relatively small 
number of respondents relied only on a few types of the information. 
This seems to suggest that, in general, public agencies tend to select the 
vendors on a comprehensive basis. 
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TABLE 15 
Number of Information Items Required in QBS Selecting Process 
(N=303) 

# of information items that an Frequency Percentages 

agency requires 
None 16 5.3 
One 4 1.3 
Two 7 2.3 
Three 6 2.0 
Four 17 5.6 
Five 25 8.3 
Six 41 13.5 
Seven 48 15.8 
Eight 42 13.9 
Nine 96 31.9 
Missing data 1 0.3 
Total 303 100% 


Negotiation after initial OBS selection 


Table 16 shows that both groups of respondents negotiated the three 
areas. A higher portion of those surveyed negotiate price, followed by 
terms and conditions. They may also negotiate other areas, including 
schedule, delivery date, quality levels and features, time frame and 
payment terms, storage, future price for long term contracts, 
reimbursable costs, software license, staff, technical requirements, 
payment schedules, training and additional warranties, travel, upgraded 
quality. One also stated that it has statutory authority to negotiate all, but 
rarely does. 


TABLE 16 
Items That Public Agencies Negotiate with Vendors After Initial 
QBS Selection 


Items that is negotiated APWA members NIGP members 
N=87 N=216 

Price 68.2% 72.7% 

Terms and conditions 62.5% 67.6% 

Scope of work 53.4% 65.3% 

Others 1.1% 14.4% 
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The questionnaire sent to the APWA members (but to NIGP 
members) asks respondents to indicate what type of selection processes 
are permitted in dealing with different purchasing. The results are 
reported in Table 17. Because agencies are allowed to use multiple 
selection processes for any type of procurements, the total for the column 
and the row is not 100%. It is noted that all the selection processes are 
permitted for use in all types of procurement, but some are permitted in 
more agencies that others. Formal competitive bid is still the widely 
permitted way for all types of purchases, except professional services, 
followed by informal quotations. Of the respondents, 23 to 42 percent 
indicate they are allowed to use the best value approach/best value 
delivery for all types of procurements. Eighty percent of responses 
stated that they use RFP for professional service, and 34 percent to 50 
percent indicate they use RFP for all other purchases. When considering 
procurement of professional services, a much higher proportion of 


TABLE 17 
Permission to Use Various Selection Processes for the Different 
Types of Procurement: Percentage of APWA Responses 


(N= 87) 

GQ) (2) |G) |@) |G) (6 
Formal competitive bid 81.8 |76.1 {51.1 [26.1 |50.0 [86.4 
Informal quotations 73.9 {52.3 |48.9 [26.1 |43.2 [43.2 
Best value approach/ best value 38.6 |31.8 |42.0 |22.7 [35.2 |25.0 

delivery 

Sole source 50.0 [37.5 |46.6 |21.6 |40.9 {19.3 
RFP 43.2 |50.0 |81.8 |34.1 [48.9 |35.2 
Two-step approach 15.9 {17.0 |27.3 |19.3 |20.5 [18.2 
Letter of interest 15.9 {18.2 |43.2 |18.2 |23.9 [15.9 
Brooks bill approach or QBS 8.0 [11.4 |35.2 {17.0 |19.3 |12.5 
Request to negotiate 11.4 {12.5 |25.0 |12.5 |19.3 [15.9 
Others 2.3 [L1 {23 |1l.1 [34 [Ll 


Notes: Column (1) = Commodities supplies, or consumables; Column (2) = 
Contractual services (i.e., janitorial, etc.); Column 3 = Professional services, 
excluding IT; Column (4) = Client service (vocational rehabilitation, etc.); 
Column (5) = Information technology; (6) = Column (6) = Construction 
contracts. 
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respondents confirm their use of letter of interest, Brooks bill, and 
request to negotiate. IT procurement runs second in using these QBS 
approaches. 


Satisfaction with OBS process 


The respondents were also asked how satisfied they are with their 
qualifications-based selection process, where 1 means “not satisfied at 
all” and 5 means “very satisfied”. The finding is reported in Table 18. 
Overall, both groups indicate that they are satisfied by their qualification- 
based process, as seen by the means of larger than 3. The mean for the 
APWA group (3.87) is higher than the NIGP group (3.42), and a larger 
proportion of APWA group rank their level of satisfaction level at “4” 
and “5” than its NIGP counterparts. 


TABLE 18 
Public Agency Satisfaction with the QBS Selection Process 


1 2 3 4 5 Missing |Mean |St. Dev 


NIGP (N=216) [5.6% {13% |23% |24% [18% |17% 3.42 |1.18 


APWA (N=87) [2.3% [3% —|23% |34% [26% |11% 3.87 | .967 


Reasons for not adopting OBS 


When asked if they have not adopted QBS, what the reasons are, 
nearly all the reasons that we listed are recognized by a small number of 
the respondents (see Table 19). In the NIGP group, the number one 


TABLE 19 
Reasons for Not Adopting QBS 

Reasons Frequency Frequency 
among APWA | among NIGP 
respondents respondents 

We do not think it is a viable method 0 3 

It takes too much time 3 7 

It is prohibited by statute or regulations 6 31 

We are satisfied with the traditional lowest |8 22 

responsible and responsive bidder 
We do not have personnel to implement it [7 16 
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reason is that law prohibits it. The next reasons are their satisfaction 
with the traditional lowest responsible and responsive bidding and the 
lack of personnel. Although the ranking differs, these three factors are 
also cited more frequently than other reasons in the APWA group. 


Likelihood for More Adoption of QBS 


The authors are also interested in the trend for QBS in the near future. 
Among the 24 APWA members that answered the question, over 58 % 
ranked the likelihood as “1” and “2”, 29% ranked “3” and 12.5% marked 
“4” and “5” (Table 20). With the mean of 2.25, it seems that the 
likelihood for many agencies to adopt QBS remains moderate. This 
information was not collected from the NIGP group due to the authors’ 
coding error. 


TABLE 20 
The Likelihood to QBS in the Coming 3-4 Years 
(1 means “not likely at all” and 5 means “very likely”) 


N= 24 1 2 3 4 5 Mean St. Dev. 
% of 33% | 25% | 29% | 8% | 4% | 2% 1.152 
Respondents 

Summary 


This section presents the findings about the use of QBS and other 
non-traditional selection methods in the public agencies from various 
aspects. The findings indicate that about 70% of the respondents report 
they select vendors based on qualifications. Two factors are found to be 
Statistically significant in explaining the use of QBS. They are the region 
variable and the enabling legislation variable. Most QBS users are in the 
south. The findings support the literature that the QBS is being used in 
many public agencies. This is particularly true with the purchasing of 
professional services and IT. In general, public agencies are happy with 
their QBS process. It is also interesting to compare the two groups (e.g., 
APWA and NIGP) in terms of how they use QBS and other non- 
traditional selection methods. In general, they report similar trend and 
process, though they do vary somewhat in how they use them. 
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CONCLUSION 


Public procurement has evolved into a strategic position in improving 
public services. While various changes have occurred in public 
procurement, this paper focuses on the use of QBS and other non- 
traditional methods by the public agencies for selecting vendors. Despite 
the potential flaws in the data due to the use of slightly different wording 
in survey questions and slightly different survey methods (online vs. 
mailed questionnaires), the authors believe that the following 
conclusions accurately depict the frequency and acceptance of QBS as a 
procurement method. 


Though the traditional lowest responsive and responsible bidding is 
still the dominant selection method when all procurement is considered, 
various non-traditional methods have been introduced to public 
procurement due to the increasing complex nature of the commodities 
and services that government purchases. This study shows that QBS has 
gained wide acceptance and use in the pubic agencies. When 
considering only professional services, QBS becomes the most common 
method. 


The paper also illustrates how public agencies are using non- 
traditional methods, particularly QBS. However, we have only very 
limited data to predict the trend in the near future. Future research can 
contribute to this area with more in-depth studies such as using 
interviews and more defined questions. Other related and important 
issues that need research include the impact of QBS and problems in 
using QBS such as possible legal disputes. 


From the policy perspective, this study has found that the use of QBS 
is statistically correlated with enabling legislature. Therefore, for public 
agencies to use QBS, legislators should grant this permission. Finally, 
several non-QBS users state that they do not have enough personnel to 
implement QBS. So public agencies need to address staffing capacity to 
ensure QBS will be implemented. 
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NOTES 


1. According to Thai and Grimm (2000), the origin of public 
procurement can be traced to 2400-2800 B.C. in Syria. As far as its 
size 1s concerned, American Bar Association (2000) reported that 
only “state and local governments were spending approximately 
$750 billions annually in the procurement of goods, supplies, 
equipment, services, and construction” (p. v). 


2. Kelman (1990) defines equity as providing fair access to bidders in 
competing for government business; integrity as reducing the 
chances for corruption in the procurement process; economy and 
efficiency as procuring at the lowest possible price for goods or 
services of the quality desired. 


3. Government procures different categories of services, and 
professional service is only of them. The other services include 
personal service (e.g., translation, technical editing, and technical 
appraisal), client services (e.g., vocational rehabilitation, guidance 
counseling, employment services), managerial services, and high 
technology services. Different selection methods are used for the 
procurement of different services, although competitive sealed 
proposal and two-step competitive sealed bidding are often used. 
While many jurisdictions authorize public agencies to use QBS to 
procure certain professional services, NASPO (1997) mentioned that 
QBS is most likely to be instituted in purchase of client service. 


4. Office of Management and Budget did not specify the current 
professional services contracting which is difficult to generalize 
given the diversification of the professional services needed by the 
federal government. 


5. The next section explains the number of state and local governments 
that have laws to authorize the use of QBS. American Council of 
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Engineering Companies maintains QBS website www.acec.org. 
This site provides a great deal information about the use of QBS. 


6. The differences are mentioned and reported in the “finding” section. 


7. For the question — which of the following selection methods is 
allowable under your enabling legislature? a) use of qualification- 
based selection method(s), b) selection of a vendor based on “the 
best interest of jurisdictions,” c) all procurements are based on low 
responsive and responsible bidder only - the online survey allows 
only a single entry while the mail survey allows multiple entries. For 
the question -- if you have not adopted qualification-based selection 
process, how likely will you adopt it in the coming 3-4 years? -- due 
to a coding error in the online survey, the information about the 
likelihood of adopting QBS by more agencies was collected only in 
the mail survey, but not in the online survey. 
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